I've confirmed the shadow maps do look properly now in Splinter Cell. As for Splinter Cell: Pandora Tomorrow (the second game in the series), it does not appear to need this hack at all. Its shadow maps seem fine with just the modified dref scale, which we already apply on master.

I guess we'd need someone to play Pandora Tomorrow more extensively to be sure, but it's not going to be me, as I'm not a fan of stealth games.

In any case, this seems fine for now 👍 .

Could the accompanying sample help? There was also one for opengl (paper), which was then ported back to dx too.
(also I think the xbox guys may have had similar doubts? and I found some old snippet about that)

Could the accompanying sample help? There was also one for opengl (paper), which was then ported back to dx too.

I don't see any magic flags/states used anywhere in those samples, so I'm afraid it doesn't add anything to what we already know.

Edit: @AlpyneDreams might be noteworthy to remark that the sample is setting:

	d3dDevice->SetTextureStageState(0, D3DTSS_TEXTURETRANSFORMFLAGS,
									D3DTTFF_COUNT4 | D3DTTFF_PROJECTED);

As one would expect. It's still odd Splinter Cell doesn't use D3DTTFF_PROJECTED.

(also I think the xbox guys may have had similar doubts? and I found some old snippet about that)

Neither seem related to Nvidia (hardware) shadow maps though, which is really the topic at hand here.

I believe the shadows are a direct port of the Xbox version and that's why they only worked on very specific hardware that was similar enough to the Xbox at the time.

@WinterSnowfall Specifically, the game's readme says this:

Why does Splinter Cell have a special mode for NV2x/NV3x graphic chips?

Splinter Cell was originally developed on XBOX™. Features only available on NV2x chips
were used and it was decided to port them to the PC version even if these chips would be the
only one able to support them. Considering the lighting system of XBOX™ was well validated,
it was easy to keep that system intact.

Splinter Cell Dynamic lighting system

Splinter Cell shadow system is a major part of the game. On NV2x/NV3x hardware, it runs
using a technique called Shadow Buffers. This technique is rendering the scene from every
shadow casting light and store a depth buffer that represent each pixel viewed by this light
source. Each pixel has an X, Y, Z coordinate in the light system and these coordinates can be
transformed, per pixel, in the viewer coordinate system. It’s then easy to compare with the actual
depth stored in the Z buffer to figure out if the pixel viewed by the camera is the same or is
occluded by the pixel viewed by the light. If they are the same, it means the pixel is lighted, if
the light pixel is in front of the viewer pixel, it means the pixel is in the shadow. On all other
current hardware, the game is using another technique called projected shadows

Mhh so.. It appears indeed like dege also reported some kind of baffling situation, even though in the following page he claimed to have discovered some whatever proper quirk.
I even tried to check UD3DRenderDevice::SupportsShadowBuffer in Ghidra, but I have hardly ever seen a more badly decompiled function (not gonna lie though, I saw a 0x200 value in the first conditional which is the value for D3DFMT_D24S8 and I lighted up).

https://developer.download.nvidia.com/GPU_Programming_Guide/GPU_Programming_Guide.pdf#page=53
Holy fettucini baloney, I think I have scored.

I think I have scored.

Early NVIDIA drivers (version 45.23 and earlier) implicitly assumed that shadow maps were to be projected. This behavior changed with NVIDIA drivers 52.16 and later—programmers now need to explicitly set the appropriate texture stage flags.

Similarly, when using the ps1.1-1.3 shader models, drivers version 52.16 and later now require that the projective flag is explicitly set for the texture-stage sampling the shadow-map (for example, SetTextureStageState(0, D3DTSS_TEXTURETRANSFORMFLAGS, D3DTTFF_PROJECTED).

Well that settles it, yes. It was a driver hack, one that was most likely deprecated between the two games, which is why Pandora Tomorrow doesn't appear to need it. Nice find.

@AlpyneDreams Might be worth updating the comments to match these findings, so that we don't forget why the hack is needed. Otherwise, it looks even more justified now.

@WinterSnowfall This is excellent news!

However it doesn't quite explain how the second sampler for the light cookie is also projected because as far I'm aware that texture isn't a depth texture...

However it doesn't quite explain how the second sampler for the light cookie is also projected because as far I'm aware that texture isn't a depth texture...

I'm assuming it's also part of the driver hack that was applied on shadow maps? Regardless, a config option sounds like the proper way to handle this exceptionally distinct behavior that unceremoniously got disabled at some point. I'm curious if we'll ever run across other games that depend on it, not necessarily with shadow maps, but relying on depth textures being projected without the D3DTTFF_PROJECTED flag.

This is all about talking about cookies.
https://web.archive.org/web/20030624134401/http://developer.nvidia.com/view.asp?IO=gdc_projshadows

Then.. indeed it appears like that change tracks with the behaviour reported with the old hardware (even though it appears like the same driver could have different behaviour depending on the hardware or some other knob?) but given it happened in late 2003 I was wondering: couldn't you just keep the old "careless" way for all the d3d8 games?
Presumably the majority of them was in fact tested and validated with that in mind.

couldn't you just keep the old "careless" way for all the d3d8 games? Presumably the majority of them was in fact tested and validated with that in mind.

Given how d3d8 games were still being released up until 2009 back in those days, and given the amount of d3d8 remasters showing up even in recent times, I don't see why we'd do that. Remember this was an Nvidia exclusive thing for GeForce 3/4 cards after all, not general d3d8 behavior.

Given how d3d8 games were still being released up until 2009 back in those days,

If you are talking about source (or UE2) still having a "long trail" of releases, I don't think anybody was doing any testing anymore by then. I guess you could argue that even in 2005 one quarter of players was still on dx8 cards then.. but like I don't know.
Especially considering that AFAIU a lot of them eventually turned into just using older shader versions but inside of d3d9.

Remember this was an Nvidia exclusive thing for GeForce 3/4 cards after all, not general d3d8 behavior.

Yesn't. In the words of the lead programmer at ubisoft that I linked above, NV20 was the standard for DX8.
Like a year by a year if not even more considering devkits.
And if not any for this, in an era of so fast pacing (PS 2.0 hardware eventually took another 10 months to be released, but microsoft was already officially out with dx9 just 2 months later) I had these qualms.
Also.. the expectation wasn't about literally every single d3d8 game, but just those doing anything fancy with shadow mapping and those few formats.

the amount of d3d8 remasters showing up even in recent times

You really don't want to think to that, given that nowadays you would test them against W10/11 meaning that not only everybody may have forgotten about old hacks.. But even the supposed specification is an afterthought.

If you are talking about source (or UE2) still having a "long trail" of releases, I don't think anybody was doing any testing anymore by then.

Not entirely sure what you mean here. We already know Pandora Tomorrow, which was released in 2004, didn't rely on the hack, so there definitely was testing being done.

Yesn't. In the words of the lead programmer at ubisoft that I linked above, NV20 was the standard for DX8.

Personal opinions aside, it really wasn't. ATI had a rather strong offering and presence on the market at the time and no hardware was the default for anything really (on PC), but of course certain companies leaned one way or another, just like it's the case today at times.

Also.. the expectation wasn't about literally every single d3d8 game, but just those doing anything fancy with shadow mapping and those few formats.

Sure, but we'll treat those on a case by case basis. You did however ask, I quote: "couldn't you just keep the old "careless" way for all the d3d8 games?"

You really don't want to think to that, given that nowadays you would test them against W10/11 meaning that not only everybody may have forgotten about old hacks.. But even the supposed specification is an afterthought.

Which is why this should really be in d3d8 dxvk what it was back in the day, a hack applied only in certain situations, where games actually rely on it. Otherwise, keeping in line with W10/11 behavior in d3d8 (and d3d9, of course) is generally for the best.

Not entirely sure what you mean here. We already know Pandora Tomorrow, which was released in 2004, didn't rely on the hack

Of course (putting aside that I think technically speaking the whole thing would still somewhat count as an off-spec hack). But forcing the prospective where the prospective is already set, is a null operation.
What I was in fact wondering was the other way around.. that is, d3d8 games that (once gotten to the point of setting up that long pointed list) don't set D3DTTFF_PROJECTED AND expects that to matter.

but of course certain companies leaned one way or another, just like it's the case today at times.

I suppose that working at nvidia before joining ubisoft may have slightly biased him, but he seems to have a point. NV20 was released 9 months before R200.
And maybe that may not have been enough to sway the market yet (in fact he also points out that with dx9 the situation was inverted) but it neatly explains why developers felt authorized to violate conventions.

Sure, but we'll treat those on a case by case basis.

I don't think even nvidia did that (then of course, if eventually it could not be possible to have the cake and eat it too, so be it.. but whether that is too optimistic or not is what I'd be trying to assess).
FWIW I just tried to grep for this game in a bunch of old drivers (44.03, 45.23, 61.71, 81.98), and while there is in fact a profile for splintercell.exe it's just normal AA or SLI bits.

You did however ask, I quote: "couldn't you just keep the old "careless" way for all the d3d8 games?"

Keep the old careless way of doing perspective divide in the sampling of shadow maps made with one of those 3 formats in ps1.0-1.3 games using d3d8. Is it better this way?
I'll totally grant that it feels dirty, but on the other hand from the bottom of my ignorance it seems a situation complex and specific enough that I don't think many people would do all that stuff if they weren't interested to this very technique.
A set that gets even smaller if again AFAICT most tended to drop d3d8 for d3d9 + downlevels.

Of note both that SCPT is a ps1.4 game, and also that while the updated 2004 programming guide actually notes that the flags should be explicitly set for ps_1.0-1.3.. the part where they say that "the driver requires it now" only mentions ps1.1-1.3.

Otherwise, keeping in line with W10/11 behavior in d3d8 (and d3d9, of course) is generally for the best.

?? Not at all? You want the original d3d8 and d3d9 functioning, not the lame modern one (which in case of d3d8 is even more stupid, because they are simply recycling d3d9).
Like, it's not even funny to point out just how many games were made then as opposed to now (if and when new-old games are even released with the original renderer).

I don't think even nvidia did that

Of course they didn't, since it was applied universally until a certain point. However, if games actually relied on the hack there would be clear indication something is off, as we had with Splinter Cell. It would also only affect games that had a Nvidia-specific path, since the same expectations would not be true on ATI.

Realistically, I doubt there were many (if any) other such games, otherwise we'd have noticed by now.

?? Not at all? You want the original d3d8 and d3d9 functioning, not the lame modern one

Where that makes sense, sure. Much like with the dref adjustment. But keep in mind the point is to also have compatibility with later games and also games released today. While this is less of a problem in d3d8, it is definitely a thing to keep in mind for d3d9.

TLDR, having it under a config option to be applied sparingly will do just fine for now and probably also in the long run.

Of course they didn't, since it was applied universally until a certain point.

I meant after that too, as I said I also checked the newer drivers.
And insofar as at least older cards still continued to work fine afterwards, and that I couldn't find anything game-specific in the drivers (I also just tried to open 61.71's nv4_disp.dll in ghidra but again to no avail) then it kinda felt like they didn't have quirks.
For instance, that d3d9-tagged guidance may just apply (indeed) to just d3d9. Or perhaps only to dx9 native generations of cards.

Realistically, I doubt there were many (if any) other such games, otherwise we'd have noticed by now.

I mean, with splinter cell it's such a cause célèbre that somebody even made up that ubisoft isn't selling pandora tomorrow due to this (which I don't know, it always rubbed my nose considering there are fare more broken games that are still sold nowadays).
But even because the supposed fallback (projector shadows) is also broken IIRC.

While this is less of a problem in d3d8, it is definitely a thing to keep in mind for d3d9.

Not even that honestly. You are talking about a few games released every year (if even as I was saying, just look at the mass effect or the sims remasters) against thousands or tens of thousands.
Then it's quite different with my proposed wondered to-be-defined d3d8 hack, but I seriously don't even know how people could look at W10 as a reference for older stuff.

I mean, with splinter cell it's such a cause célèbre that somebody even made up that ubisoft isn't selling pandora tomorrow due to this

That particular story isn't made up at least. Pandora Tomorrow does not render any light sources or shadows whatsoever without a proper dref scale (another behavior that was changed unceremoniously at some point), and gameplay/the stealth mechanics directly depend on the player being able to pick up on those cues and stay out of detection. It's impossible to play it (enjoyably) on modern systems because of that, and I guess Ubisoft has decided it's not worth fixing.

In the original game, the projected fallback mode still "works" as far as gameplay is concerned, but obviously more subtle effects will be missing. It's an inconvenience, but it does not break gameplay, and owners of ATI cards would have had the same experience even back in the day.

My memory is that I should have played it in around 2009, with what must have been a HD 2600 XT. Either I was just so young and dumb that I pushed through it with just sheer will and incredible suspension of disbelief (you can find indeed plenty of people on the net claiming they didn't mind beating the game with their GTX 960), or that was actually still working back then (and the guy that got in touch with phil in 2014 to tell him R600 was irredeemably broken, was on too new drivers).
FWIW I looked yet again for strings in older drivers, and while ati did in fact have a mention for every splinter cell games executable in the 9.7's ati3duag.dll I couldn't ascertain the purpose.